THE DEVIATION MATRIX OF A CONTINUOUS-TIME MARKOV CHAIN

The deviation matrix of a continuous-time Markov chain

The deviation matrix of an ergodic, continuous-time Markov chain with transition probability matrix P (.) and ergodic matrix Π is the matrix D ≡ ∫∞ 0 (P (t)−Π)dt. We give conditions for D to exist and discuss properties and a representation of D. The deviation matrix of a birth-death process is investigated in detail. We also describe a new application of deviation matrices by showing that a me...

The Deviation Matrix of Continuous - Time Markov Processes - Theory and Applications

An Approximation Approach for the Deviation Matrix of Continuous-Time Markov Processes with Application to Markov Decision Theory

We present an update formula that allows the expression of the deviation matrix of a continuous-time Markov process with denumerable state space having generator matrix Q∗ through a continuous-time Markov process with generator matrix Q. We show that under suitable stability conditions the algorithm converges at a geometric rate. By applying the concept to three different examples, namely, the ...

An Imprecise Probabilistic Estimator for the Transition Rate Matrix of a Continuous-Time Markov Chain

We consider the problem of estimating the transition rate matrix of a continuous-time Markov chain from a finite-duration realisation of this process. We approach this problem in an imprecise probabilistic framework, using a set of prior distributions on the unknown transition rate matrix. The resulting estimator is a set of transition rate matrices that, for reasons of conjugacy, is easy to fi...

Efficient Continuous-Time Markov Chain Estimation

Many problems of practical interest rely on Continuous-time Markov chains (CTMCs) defined over combinatorial state spaces, rendering the computation of transition probabilities, and hence probabilistic inference, difficult or impossible with existing methods. For problems with countably infinite states, where classical methods such as matrix exponentiation are not applicable, the main alternati...